SuiteFixture Setup

The book has now been published and the content of this chapter has likely changed substanstially.

How do we cause the Shared Fixture to be built before the first test method that needs it?

We build/destroy the shared fixture in special methods called by the Test Automation Framework before/after the first/last Test Method is called.

Sketch SuiteFixture Setup embedded from SuiteFixture Setup.gif

Shared Fixtures (page X) are commonly used to reduce the amount of per-test overhead for setting up the fixture. Sharing a fixture involves extra test programmming effort because we need to create the fixture and have a way of discovering the fixture in each test. Regardless of how the fixture is accessed, it must be initialized (constructed) before it is used.

SuiteFixture Setup is one way to initialize the fixture if all the Test Methods (page X) that need it are defined on the same Testcase Class (page X).

How It Works

We implement or override a pair of methods that the Test Automation Framework (page X) calls automatically. The name or annotion of these methods varies between members of the xUnit family but all work the same way: the framework calls the SuiteFixture Setup method before it calls the setUp method for the first Test Method and it calls the SuiteFixture Teardown method after the tearDown method for the final Test Method. (I would have preferred to say "... method on the first/final Testcase Object (page X)" but that isn't true since NUnit, unlike other members of the xUnit family, only creates a single Testcase Object. See the sidebar There's Always an Exception (page X) for details.)

When To Use It

We can use SuiteFixture Setup when we have a test fixture we wish to share between all the Test Methods of a single Testcase Class and our variant of xUnit supports this feature. It is particularly useful if we need to tear down the fixture after the last test is run. At the time of writing this book, only VbUnit and NUnit and JUnit 4.0 supported SuiteFixture Setup "out of the box" but it is not difficult to add it in most variants of xUnit.

If we need to share the fixture more widely, we either have to use a Prebuilt Fixture (page X), a Setup Decorator (page X) or Lazy Setup (page X). If we don't want to share the actual instance of the fixture but we do want to share the code to set up the fixture, we can use Implicit Setup (page X) or Delegated Setup (page X).

The main reason for using a Shared Fixture and hence SuiteFixture Setup is Slow Tests (page X) caused by too many test fixture objects being created each time every test is run. Of course, a Shared Fixture can led to Interacting Tests (see Erratic Test on page X) or even a Test Run War (see Erratic Test); refer to the sidebar Faster Tests Without Shared Fixtures (page X) for other ways to solve this problem.

Implementation Notes

For SuiteFixture Setup to work properly, we must ensure that the fixture is remembered between calls to the Test Methods. This implies we need to use a class variable, Registry[PEAA] or Singleton[GOF] to hold the references to the fixture. (Except in NUnit; see the sidebar There's Always an Exception.) The exact implementation varies from one member of the xUnit family to the next; here are a few highlights:

In VbUnit, we implement the interface IFixtureFrame in your Testcase Class thereby causing the Test Automation Framework to call the IFixtureFrame_Create method before the first Test Method is called and the IFixtureFrame_Destroy method after the last Test Method is called.

In NUnit, the attributes [TestFixtureSetUp] and [TestFixtureTearDown] are used inside a TestFixture to designate the methods to be called once prior to executing any of the tests in the fixture and once after all tests are completed

In JUnit 4.0 and later, the attribute @BeforeClass is used to indicate that a method should be run once before the first Test Method is executed. The method with the attribute @AfterClass is run after the last Test Method is run. JUnit allows these methods to be inherited and overriden; the subclass' methods are run between the superclass' methods.

Just because we are using a form of Implicit Setup to invoke the construction and destruction of the test fixture doesn't mean that we should dump all the fixture setup logic into the SuiteFixture Setup. We can still use Creation Methods (page X) to move complex construction logic into places where it can be tested and reused more easily (such as a Testcase Superclass (page X) or a Test Helper (page X).)

Motivating Example

Suppose we have the following test:

   [SetUp]
   protected void setUp() {
      helper.setupStandardAirportsAndFlights();
   }

   [TearDown]
   protected void tearDown()  {
      helper.removeStandardAirportsAndFlights();
   }

   [Test]
   public void testGetFlightsByOriginAirport_2OutboundFlights(){
      FlightDto[] expectedFlights = helper.findTwoOutboundFlightsFromOneAirport();
      long originAirportId = expectedFlights[0].OriginAirportId;
      // Exercise System
      IList flightsAtOrigin = facade.GetFlightsByOriginAirport(originAirportId);
      // Verify Outcome
      AssertExactly2FlightsInDtoList( expectedFlights[0], expectedFlights[1],
            flightsAtOrigin,    "Flights at origin");
   }

   [Test]
   public void testGetFlightsByOriginAirport_OneOutboundFlight(){
      FlightDto expectedFlight = helper.findOneOutboundFlight();
      // Exercise System
      IList flightsAtOrigin = facade.GetFlightsByOriginAirport(
         expectedFlight.OriginAirportId);
      // Verify Outcome
      AssertOnly1FlightInDtoList( expectedFlight,
                  flightsAtOrigin, "Outbound flight at origin");
   }
Example ImplicitFixtureSetupCSharp embedded from CSharp/Exercise-06-SuiteFixtureSetup/Com/Clrstream/FlightBooking/Services/Test/ImplicitFlightManagementFacadeTest.cs

The following is the console generated by an instrumented version of these tests:

  -------------------- setUp setupStandardAirportsAndFlights
   testGetFlightsByOriginAirport_OneOutboundFlight
   tearDown removeStandardAirportsAndFlights
  -------------------- setUp setupStandardAirportsAndFlights
   testGetFlightsByOriginAirport_TwoOutboundFlights
   tearDown removeStandardAirportsAndFlights
  --------------------
Example ImplicitFixtureSetupConsole embedded from CSharp/ImplicitFixtureSetupConsole.txt

Note how setupStandardAirportsAndFlights is called before each Test Method. The horizontal lines delineate the test boundaries.

Refactoring Notes

Suppose we want to refactor to a Shared Fixture; if we don't care about destroying it when the test run is finished, we could use Lazy Setup. Otherwise, we can convert to a SuiteFixture Setup strategy by simply moving our code from the setUp and tearDown methods to the suiteFixtureSetUp and suiteFixtureTearDown methods respectively.

In NUnit, we use the attributes [TestFixtureSetUp] and [TestFixtureTearDown] to indicate these methods to the Test Automation Framework. If we don't want to leave anything in our setUp/tearDown methods we can simply change the attributes from [Setup] and TearDown to [TestFixtureSetUp] and [TestFixtureTearDown], respectively.

Example: SuiteFixture Setup

Here's the result of our refactoring to SuiteFixture Setup:

 [TestFixtureSetUp]
   protected void suiteFixtureSetUp()
   {
      helper.setupStandardAirportsAndFlights();
   }

   [TestFixtureTearDown]
   protected void suiteFixtureTearDown()  {
      helper.removeStandardAirportsAndFlights();
   }

   [SetUp]
   protected void setUp() {
   }

   [TearDown]
   protected void tearDown()  {
   }

   [Test]
   public void testGetFlightsByOrigin_TwoOutboundFlights(){
      FlightDto[] expectedFlights = helper.findTwoOutboundFlightsFromOneAirport();
      long originAirportId = expectedFlights[0].OriginAirportId;
      // Exercise System
      IList flightsAtOrigin = facade.GetFlightsByOriginAirport(originAirportId);
      // Verify Outcome
      AssertExactly2FlightsInDtoList( expectedFlights[0], expectedFlights[1],
            flightsAtOrigin,    "Flights at origin");
   }

   [Test]
   public void testGetFlightsByOrigin_OneOutboundFlight() {
      FlightDto expectedFlight = helper.findOneOutboundFlight();
      // Exercise System
      IList flightsAtOrigin = facade.GetFlightsByOriginAirport(
         expectedFlight.OriginAirportId);
      // Verify Outcome
      AssertOnly1FlightInDtoList( expectedFlight,
                  flightsAtOrigin, "Outbound flight at origin");
   }
Example SuiteFixtureSetupCSharp embedded from CSharp/Exercise-06-SuiteFixtureSetup/Com/Clrstream/FlightBooking/Services/Test/FlightManagementFacadeTest.cs

The console showing when various methods of the Testcase Class are called now looks like this:

suiteFixtureSetUp setupStandardAirportsAndFlights
  -------------------- setUp
   testGetFlightsByOriginAirport_OneOutboundFlight
   tearDown
  -------------------- setUp
   testGetFlightsByOriginAirport_TwoOutboundFlights
   tearDown
  --------------------
suiteFixtureTearDown removeStandardAirportsAndFlights
Example SuiteFixtureSetupConsole embedded from CSharp/SuiteFixtureSetupConsole.txt

Note that the setUp method is still called before each Test Method in addition to the suiteFixtureSetUp method where we are now calling setupStandardAirportsAndFlights to set up our fixture. So far, this is no different than Lazy Setup; where the difference lies is that removeStandardAirportsAndFlights is called after the last of our Test Methods.

About the Name

Naming this pattern was tough because each variant of xUnit that implements it has a different name for it. Complicating matters is the fact that the Microsoft camp uses test fixture to mean more than what the Java/Pearl/Ruby/... camp means. I landed on SuiteFixture Setup by focusing on the scope of the Shared Fixture; it is shared across the test suite for one Testcase Class that spawns a single Test Suite Object (page X). The fixture that is built for the Test Suite Object could be called a "SuiteFixture". I hope it works for you.